Error Forgetting of Bregman Iteration

This short article analyzes an interesting property of the Bregman iterative procedure, which is equivalent to the augmented Lagrangian method, for minimizing a convex piece-wise linear function J(x) subject to linear constraints Ax = b. The procedure obtains its solution by solving a sequence of unconstrained subproblems of minimizing J(x)+ 12‖Ax−b ‖2, where b k is iteratively updated. In practice, the subproblems can be solved at relatively low accuracy. Let w denote the numerical error at iteration k. If all w are sufficiently small so that Bregman iteration identifies the optimal face, then on the optimal face, Bregman iteration enjoys an interesting error-forgetting property: the distance between the current point x̄ and the optimal solution set X∗ is bounded by ‖w − w‖, independent of the numerical errors at previous iterations. This property partially explains why the Bregman iterative procedure works well for sparse optimization and, in particular, `1minimization. The error-forgetting property is unique to piece-wise linear functions (i.e., polyhedral functions) J(x), and the results of this article appears to new to the literature of the augmented Lagrangian method.